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CLAIMS 

What is claimed is: 

1 . A method for identifying correspondence between an original video sequence comprising a 
plurality of original frames and a processed video sequence comprising a plurality of processed frames, the 
method comprising: 

(a) dividing the processed video sequence into a plurality of processed sets, each processed set having 
one or more processed frames; 

(b) identifying, for each processed set, one or more original sets from the original video sequence, 
wherein: 

each original set comprises one or more original frames; and 

two or more original sets are identified for at least one processed set; 

(c) generating a mapping for each original set corresponding to each processed set, wherein: 
the mapping defines, for the original set, a mapped set that approximates the corresponding 

processed set; and 

the mapping minimizes a local prediction error between the mapped set and the corresponding 
processed set; and 

(d) selecting, for each processed set, the original set whose mapping minimizes an accumulated 
prediction error for the processed video sequence. 

2. The invention of claim 1, wherein: 

each original set consists of one or two original frames; 
each mapped set consists of a single mapped frame; and 
each processed set consists of a single processed frame. 

3. The invention of claim 1, wherein generating each mapping involves temporal registration. 

4. The invention of claim 3, wherein generating each mapping further involves at least one of spatial 
registration and histogram registration. 

5. The invention of claim 4, wherein generating each mapping involves spatial registration and 
histogram registration. 

6. The invention of claim 5, wherein generating each mapping involves minimizing the local 
prediction error using processing that (1) treats temporal, spatial, and histogram registration as three 
parameter sets, (2) fixes two of the three parameter sets, while optimizing the third, and (3) iterates on all 
three parameter sets until an optimal solution is found. 
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7. The invention of claim 3, wherein the temporal registration involves minimizing the local 
prediction error between a weighted sum of two original frames and a corresponding processed frame. 

8. The invention of claim 1, wherein one or more constraints are applied to limit the number of 
different original sets that are identified for each processed set. 

9. The invention of claim 8, wherein at least one constraint is a causal constraint that specifies that no 
original frames displayed in the past can be used to generate a current processed frame. 

10. The invention of claim 1, wherein: 

a first selected original set corresponds to the first processed frame in the processed video stream; and 
each other selected original set for each other processed frame depends on a selected original set 
corresponding to a previous processed frame. 

1 1 . The invention of claim 10, wherein the dependence between selected original sets is based on one 
or more constraints. 

12. The invention of claim 1 1, wherein at least one constraint is a causal constraint that specifies that 
no original frames displayed in the past can be used to generate a current processed frame. 

13. The invention of claim 1 , wherein at least one original frame is not included in any mapping. 

14. The invention of claim 1, wherein one processed frame is a repetition of the immediately 
preceding processed frame. 

15. The invention of claim 1, wherein: 

each original set is selected based on a causal constraint that a matching index for each processed set 
conforms to a monotonically increasing function; 

the matching index identifies the original frame in each original set having the largest frame index; and 
frame indices increment by one from frame to frame in the original video sequence. 

16. The invention of claim 1, wherein: 

the local prediction error between each mapped set and the corresponding processed set is a function 
of a matching term and a context term; 

the matching term characterizes differences between the mapped set and the corresponding processed 
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set; and 

the context term corresponds to a cost associated with one or more contextual constraints applied to a 
temporal relationship between the original and processed video sequences. 

17. The invention of claim 16, wherein the matching term corresponds to a mean squared error 
between the mapped set and the corresponding processed set. 

18. The invention of claim 16, wherein the context term enforces a causal constraint that no previously 
displayed frames can be subsequently processed. 

19. The invention of claim 16, wherein the context term penalizes repetition and dropping of original 
frames. 

20. The invention of claim 16, wherein: 

the context term encourages an assumption that a matching index increments by one from frame to 
frame in the processed video sequence; 

the matching index identifies the original frame in each original set having the largest frame index; and 
frame indices increment by one from frame to frame in the original video sequence. 

21. The invention of claim 1, wherein the processed video sequence has been generated by capturing 
with a camcorder a display of the original video sequence. 

22. The invention of claim 1, further comprising the steps of: 

(e) identifying a watermark in an original frame; and 

(f) determining whether a corresponding processed frame has a similar watermark. 

23. The invention of claim 1, further comprising the step of adjusting the processed video sequence to 
correct for one or more misalignments between the original and processed video sequences. 

24. The invention of claim 23, wherein the one or more misalignments include at least one of spatial 
misalignment and histogram misalignment. 

25. The invention of claim 24, wherein the one or more misalignments include both spatial 
misalignment and histogram misalignment. 

26. A machine-readable medium, having encoded thereon program code, wherein, when the program 
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code is executed by a machine, the machine implements a method for identifying correspondence between 
an original video sequence comprising a plurality of original frames and a processed video sequence 
comprising a plurality of processed frames, the method comprising: 

(a) dividing the processed video sequence into a plurality of processed sets, each processed set having 
one or more processed frames; 

(b) identifying, for each processed set, one or more original sets from the original video sequence, 
wherein: 

each original set comprises one or more original frames; and 

two or more original sets are identified for at least one processed set; 

(c) generating a mapping for each original set corresponding to each processed set, wherein: 
the mapping defines, for the original set, a mapped set that approximates the corresponding 

processed set; and 

the mapping minimizes a local prediction error between the mapped set and the corresponding 
processed set; and 

(d) selecting, for each processed set, the original set whose mapping minimizes an accumulated 
prediction error for the processed video sequence. 

27. An apparatus for identifying correspondence between an original video sequence comprising a 
plurality of original frames and a processed video sequence comprising a plurality of processed frames, the 
apparatus comprising: 

(a) means for dividing the processed video sequence into a plurality of processed sets, each processed 
set having one or more processed frames; 

(b) means for identifying, for each processed set, one or more original sets from the original video 
sequence, wherein: 

each original set comprises one or more original frames; and 

two or more original sets are identified for at least one processed set; 

(c) means for generating a mapping for each original set corresponding to each processed set, 
wherein: 

the mapping defines, for the original set, a mapped set that approximates the corresponding 
processed set; and 

the mapping minimizes a local prediction error between the mapped set and the corresponding 
processed set; and 

(d) means for selecting, for each processed set, the original set whose mapping minimizes an 
accumulated prediction error for the processed video sequence. 



